Natural Scene Image Modeling Using Color and Texture Visterms

This paper presents a novel approach for visual scene representation, combining the use of quantized color and texture local invariant features (referred to here as visterms) computed over interest point regions. In particular we investigate the different ways to fuse together local information from texture and color in order to provide a better visterm representation. We develop and test our methods on the task of image classification using a 6-class natural scene database. We perform classification based on the bag-of-visterms (BOV) representation (histogram of quantized local descriptors), extracted from both texture and color features. We investigate two different fusion approaches at the feature level: fusing local descriptors together and creating one representation of joint texture-color visterms, or concatenating the histogram representation of both color and texture, obtained independently from each local feature. On our classification task we show that the appropriate use of color improves the results w.r.t. a texture only representation.